Search results for " Bregman divergence."

showing 4 items of 4 documents

Extropy: Complementary Dual of Entropy

2015

This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a complementary dual function which we call "extropy." The entropy and the extropy of a binary distribution are identical. However, the measure bifurcates into a pair of distinct measures for any quantity that is not merely an event indicator. As with entropy, the maximum extropy distribution is also the uniform distribution, and both measures are invariant with respect to permutations of their mass functions. However, they behave quite differently in their assessments…

Bregman divergenceFOS: Computer and information sciencesStatistics and ProbabilitySettore MAT/06 - Probabilita' E Statistica MatematicaKullback–Leibler divergenceComputer Science - Information TheoryGeneral MathematicsFOS: Physical sciencesBinary numberMathematics - Statistics TheoryStatistics Theory (math.ST)Kullback–Leibler divergenceBregman divergenceproper scoring rulesGini index of heterogeneityDifferential entropyBinary entropy functionFOS: MathematicsEntropy (information theory)Statistical physicsDual functionAxiomMathematicsdifferential and relative entropy/extropy Kullback- Leibler divergence Bregman divergence duality proper scoring rules Gini index of heterogeneity repeat rate.Settore ING-INF/05 - Sistemi Di Elaborazione Delle InformazioniDifferential and relative entropy/extropyInformation Theory (cs.IT)Probability (math.PR)repeat ratePhysics - Data Analysis Statistics and ProbabilitydualityStatistics Probability and UncertaintySettore SECS-S/01 - StatisticaMathematics - ProbabilityData Analysis Statistics and Probability (physics.data-an)Statistical Science
researchProduct

The on-line curvilinear component analysis (onCCA) for real-time data reduction

2015

Real time pattern recognition applications often deal with high dimensional data, which require a data reduction step which is only performed offline. However, this loses the possibility of adaption to a changing environment. This is also true for other applications different from pattern recognition, like data visualization for input inspection. Only linear projections, like the principal component analysis, can work in real time by using iterative algorithms while all known nonlinear techniques cannot be implemented in such a way and actually always work on the whole database at each epoch. Among these nonlinear tools, the Curvilinear Component Analysis (CCA), which is a non-convex techni…

Clustering high-dimensional dataBregman divergenceComputer scienceneural networkprojectionBregman divergenceNovelty detectionSynthetic dataData visualizationArtificial Intelligencebranch and boundComputer visionunfoldingcurvilinear component analysisCurvilinear coordinatesArtificial neural networkbusiness.industryVector quantizationPattern recognitiononline algorithmbearing faultvector quantizationPattern recognition (psychology)Principal component analysisbearing fault; branch and bound; Bregman divergence; curvilinear component analysis; data reduction; neural network; novelty detection; online algorithm; projection; unfolding; vector quantization; Software; Artificial Intelligencedata reductionArtificial intelligencebusinessnovelty detectionSoftware
researchProduct

Coherent Conditional Previsions and Proper Scoring Rules

2012

In this paper we study the relationship between the notion of coherence for conditional prevision assessments on a family of finite conditional random quantities and the notion of admissibility with respect to bounded strictly proper scoring rules. Our work extends recent results given by the last two authors of this paper on the equivalence between coherence and admissibility for conditional probability assessments. In order to prove that admissibility implies coherence a key role is played by the notion of Bregman divergence.

Settore MAT/06 - Probabilita' E Statistica Matematicabregman divergenceproper scor- ing rulesConditional prevision assessmentsconditional scoring rulesstrong dominanceConditional probabilityweak dominanceCoherence (statistics)Bregman divergenceConditional prevision assessments coherence proper scoring rules conditional scoring rules weak dominance strong dominance admissibility Bregman divergence.proper scoring rulescoherenceBounded functionKey (cryptography)admissibilityConditional prevision assessments; conditional scoring rules; admissibility; proper scor- ing rules; weak dominance; strong dominanceEquivalence (measure theory)Mathematical economicsconditional prevision assessments; strong dominance; admissibility; proper scoring rules; bregman divergence; weak dominance; conditional scoring rules; coherenceMathematics
researchProduct

Coherent conditional probabilities and proper scoring rules

2011

In this paper we study the relationship between the notion of coherence for conditional probability assessments on a family of conditional events and the notion of admissibility with respect to scoring rules. By extending a recent result given in literature for unconditional events, we prove, for any given strictly proper scoring rule s, the equivalence between the coherence of a conditional probability assessment and its admissibility with respect to s. In this paper we focus our analysis on the case of continuous bounded scoring rules. In this context a key role is also played by Bregman divergence and by a related theoretical aspect. Finally, we briefly illustrate a possible way of defin…

total coherenceSettore MAT/06 - Probabilita' E Statistica Matematicabregman divergencestrong dominanceconditional scoring rulesConditional probability assessments coherence penalty criterion proper scoring rules conditional scoring rules weak dominance strong dominance admissibility Bregman divergence g-coherence total coherence imprecise probability assessments.weak dominancestrong dominance; conditional probability assessments; imprecise probability assessments; gcoherence; proper scoring rules; bregman divergence; weak dominance; coherence; imprecise probability assessments.; admissibility; g-coherence; penalty criterion; conditional scoring rules; total coherencepenalty criteriongcoherenceproper scoring rulescoherenceconditional probability assessmentsg-coherenceimprecise probability assessmentsadmissibility
researchProduct